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DETAILED ACTION 

Specification 

1 . The specification is objected to as failing to provide proper antecedent basis for the 
claimed subject matter. See 37 CFR 1.75(d)(1) and MPEP § 608.01(o). Correction of the 
following is required: the specification does not define the terms "article of manufacture" or 
"computer usable medium", which are used in claim 19, and it is unclear what they mean. See 
also objection to claim 19 below. 

Claim Objections 

2. Claim 19 is objected to because of the following informalities: the phrases "article of 
manufacture" and "computer usable medium" have not been defined in the specification, and it is 
unclear what they mean. Although reference is made to a "storage medium" (Specification, page 
17, line 16), these two terms are not mentioned. For purposes of examination, the claim and its 
dependent claim have been treated to exclude computer communications media such as signals 
or carrier waves. Appropriate correction is required 

3. 

4. Claim 21 is objected to because of the following informalities: in lines 23-25, the claim 
states that the "first, second, third, and fourth encoding algorithms are chosen to allow for ... 
compression of speech, music, video, text, and graphics respectively". However, this states that 
4 algorithms correspond to 5 types of data. It is assumed that the word "text" is inserted 
erroneously, as the body of the claim does not refer to a numbered encoding algorithm for text 
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(step (g) of the claim implies that the text is encoded, but does not make reference to an 
algorithm). Appropriate correction is required. 

Claim Rejections - 35 USC §102 

5. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2 ) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 351(a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 

6. Claims 1, 5, 6, 10-13, and 16-17 are rejected under 35 U.S.C. 102(e) as being anticipated 
by Saunders et al. (US 6,35 1,733), hereinafter referred to as Saunders. 

7. With respect to independent claim 1, Saunders teaches a system providing low 
bit-rate compression of data comprising speech and music 
components for transmission (col. 2, lines 15-18, audio program is produced so that 
the audio content is readily fabricated 'for transmission) , over a network, said 
system comprising: 

a. a speech encoder encoding said speech component via a 
first encoding algorithm, transforming said encoded speech signal 
into a format suitable for transmission (Fig. 9-12, col. 15, lines 21-24, 
signal is compressed with a speech-only codec) , and embedding synchronization 
information associated with said speech component (col. 16, lines 15-24; 
claim 19; audio tracks are time-synchronized and video-frame synchronized to a video signal) ; 
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b. a music encoder encoding said music component via a 
second encoding algorithm, said second encoding algorithm 
different from said first encoding algorithm; transforming said 
encoded music signal into a format suitable for transmission (Fig. 
9-12, col. 15, lines 21-30; non-speech signal is encoded using a general compression algorithm; 
col. 16, lines 9-12, two parallel streams are fed into two distinct compression algorithms) ; and 
embedding synchronization information associated with said music 
component (col. 16, lines 15-24; claim 19; audio tracks are time-synchronized and video- 
frame synchronized to a video signal) ; and 

c. a multiplexer multiplexing said outputs of said speech 
encoder and said music encoder for transmission over said network 
(Fig. 12, col. 16, lines 24-28, outputs of compression units are multiplexed in a specific manner 
so that the audio can be transmitted over a digital medium) , 

wherein said first and second encoding algorithms are chosen to 
allow for low bit-rate compression of speech and music 
respectively (col. 15, lines 27-30, the distinction between compression using the speech- 
only codec and the general codec helps to reduce the required bandwidth) . 

8. With respect to claim 5, Saunders teaches everything claimed, as applied above (see 
claim 1); in addition, Saunders teaches a system as per claim 1, wherein audio 
volumes associated with said speech component and said music 
component are modifiable relative to each other (col. 6, lines 44-54, the 
volume of each signal may be independently adjusted; col. 22, lines 53-67; claim 9; the ratio 
between the levels of the channels may be adjusted) . 
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9. With respect to claim 6, Saunders teaches everything claimed, as applied above (see 
claim 1); in addition, Saunders teaches a system as per claim 1, wherein said 
speech encoder is a LPC, MELP, CELP, or waveform interpolation 
encoder (col. 15, lines 24-27, speech coding can be conducted using any known speech codec 
such as the CELP codec) . 

10. With respect to claim 10, Saunders teaches everything claimed, as applied above (see 
claim 1); in addition, Saunders teaches a system as per claim 1, wherein said 
music encoder is a transform-based encoder (col. 20, lines 20-51, audio 
codecs which may be used include MP3, which is a transform-based encoder) . 

11. With respect to claim 11, Saunders teaches everything claimed, as applied above (see 
claim 1); in addition, Saunders teaches a system as per claim 1, wherein said 
network is any of the following: local area network, wide area 
network, the Internet, cellular network, storage network, or 
wireless network (col. 9, lines 37-40; col. 25, lines 23-30; transmission may be an ISDN 
transmission to a PC modem) . 

12. With respect to independent claim 12, Saunders teaches a system providing low 
bit-rate compression of audio comprising speech and music 
components for transmission over a communication channel (col. 2, 
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lines 15-18, audio program is produced so that the audio content is readily fabricated for 
transmission), said system comprising: 

a. a first analog- to-digital converter converting said 
speech component into a digital speech signal (col. 8, lines 16-43; all 
audio sources utilize microphones to transducer audio information into real-time electrical 
signals, from which digital masters are created, using a digital format such as PCM) ; 

b. a speech encoder encoding said digital speech signal via 
a first encoding algorithm (Fig. 9-12, col. 15, lines 21-24, signal is compressed 
with a speech-only codec) ; 

c. a speech audio formatter transforming said encoded speech 
signal into a format suitable for transmission (col. 16, lines 24-25, 
encoded data is ready for multiplexing and transmission) and embedding 
synchronization information associated with said speech component 
(col. 16, lines 15-24; claim 19; audio tracks are time-synchronized and video-frame 
synchronized to a video signal) ; 

d. a second analog- to-digital converter converting said 
music component into a digital music signal (col. 8, lines 16-43; all audio 
sources utilize microphones to transducer audio information into real-time electrical signals, 
from which digital masters are created, using a digital format such as PCM) ; 

e. a music encoder encoding said digital music signal via a 
second encoding algorithm, said second encoding algorithm 
different from said first encoding algorithm (Fig. 9-12, col. 15, lines 21- 
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30; non-speech signal is encoded using a general compression algorithm; col. 16, lines 9-12, 
two parallel streams are fed into two distinct compression algorithms) ; 

f. a music audio formatter transforming said encoded music 
signal into a format suitable for transmission (col. 16, lines 24-25, 
encoded data is ready for multiplexing and transmission) and embedding 
synchronization information associated with said music component 
(col. 16, lines 15-24; claim 19; audio tracks are time-synchronized and video-frame 
synchronized to a video signal) ; and 

g. a multiplexer multiplexing said outputs of said speech 
audio formatter and said music audio formatter for transmission 
over said channel (Fig. 12, col. 16, lines 24-28, outputs of compression units are 
multiplexed in a specific manner so that the audio can be transmitted) . 

13. With respect to claim 13, Saunders teaches everything claimed, as applied above (see 
claim 12); in addition, Saunders further teaches a system as per claim 12, 
wherein said speech encoder is a LPC, MELP, CELP or waveform 
interpolation encoder (col. 15, lines 24-27, speech coding can be conducted using any 
known speech codec such as the CELP codec) . 

14. With respect to claim 16, Saunders teaches everything claimed, as applied above (see 
claim 12); in addition, Saunders further teaches a system as per claim 12, 
wherein said music encoder is a transform-based encoder (col. 20, 
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lines 20-51, audio codecs which may be used include MP3, which is a transform-based 
encoder) . 

15. With respect to independent claim 17, Saunders teaches a method to encode 
audio for transmission over a communication channel, said audio 
comprising speech and music components (col. 2, lines 15-18, audio program is 
produced so that the audio content is readily fabricated for transmission) , said method 
comprising : 

a. converting said speech component into a digital speech 
signal (col. 8, lines 16-43; all audio sources utilize microphones to transducer audio 
information into real-time electrical signals, from which digital masters are created, using a 
digital format such as PCM) ; 

b. encoding said digital speech signal via a first encoding 
algorithm (Fig. 9-12, col. 15, lines 21-24, signal is compressed with a speech-only codec) ; 

c. transforming said encoded speech signal into a format 
suitable for transmission (col. 16, lines 24-25, encoded data is ready for 
multiplexing and transmission) and embedding synchronization information 
associated with said speech component (col. 16, lines 15-24; claim 19; audio 
tracks are time-synchronized and video-frame synchronized to a video signal) ; 

d. converting said music component into a digital music 
signal (col. 8, lines 16-43; all audio sources utilize microphones to transducer audio 
information into real-time electrical signals, from which digital masters are created, using a 
digital format such as PCM) ; 
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e. encoding said digital music signal via a second encoding 
algorithm, said second encoding algorithm different from said 
first encoding algorithm (Fig. 9-12, col. 15, lines 21-30; non-speech signal is 
encoded using a general compression algorithm; col. 16, lines 9-12, two parallel streams are fed 
into two distinct compression algorithms) ; 

f . transforming said encoded music signal into a format 
suitable for transmission (col. 16, lines 24-25, encoded data is ready for 
multiplexing and transmission) and embedding synchronization information 
associated with said music component (col. 16, lines 15-24; claim 19; audio 
tracks are time-synchronized and video-frame synchronized to a video signal) ; and 

g. multiplexing said outputs of steps (c) and (f) for 
transmission over said channel (Fig. 12, col. 16, lines 24-28, outputs of 
compression units are multiplexed in a specific manner so that the audio can be transmitted) . 

Claim Rejections - 35 USC §103 

16. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

17. Claim 19 is rejected under 35 U.S.C. 103(a) as being unpatentable over Saunders et al. 
(US 6,351,733), hereinafter referred to as Saunders. 

18. With respect to independent claim 19, Saunders teaches an article of 
manufacture comprising a computer usable medium having computer 
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readable program code embodied therein for decoding transmitted 
data received over a communication channel (col. 9, lines 37-40; col. 25, 
lines 23-30; transmission may be an ISDN transmission to a PC modem. While Saunders does 
not explicitly teach that the method is performed using computer readable program code 
embodied on a computer usable medium, it at least implies that it is implemented on a computer, 
since the transmission may be received by a computer) , said transmitted data 
comprising a plurality of components, each component encoded via 
a separate encoding algorithm to provide low bit-rate compression 
(col. 16, lines 9-12, two parallel audio streams are fed into two distinct compression algorithms 
during encoding) , said medium comprising: 

a. computer readable program code aiding in receiving said 
transmitted data received over said communication channel (Fig. 13, 
col. 22, lines 53-54, digital bitstream is received) ; 

b. computer readable program code de-multiplexing said data 
into a plurality of components, said components comprising at 
least a speech component and a music component (Fig. 13, col. 22, lines 54- 
55, two audio signals are separated from each other) ; 

c . computer readable program code decoding said speech 
component via a first decoding algorithm (Fig. 13, col. 22, lines 54-55, two 
audio signals are decoded to digital audio signals) ; and 

d. computer readable program code decoding said music 
component via a second decoding algorithm (Fig. 13, col. 22, lines 54-55, two 
audio signals are decoded to digital audio signals) , said second decoding 
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algorithm different from said first decoding algorithm (see 
preamble) . 

It is noted that Saunders teaches the encoding process in detail (see claim 1), and that the 
decoding process is merely the reversal of the same sequence of steps. 

19. Claim 2 is rejected under 35 U.S.C. 103(a) as being unpatentable over Saunders et al. 
(US 6,351,733), hereinafter referred to as Saunders, as applied to claim 1 above, and further in 
view of Kane et al. (US 5,293,450), hereinafter referred to as Kane. 

20. With respect to claim 2, Saunders teaches everything claimed, as applied above (see 
claim 1); but Saunders docs not explicitly teach a system as per claim 1, wherein 
said data is a composite of said speech and music components and 
said system further comprises a signal separator, said signal 
separator separating said speech and music components from said 
compos i t e . However, the examiner contends that this concept was well known, as taught by 
Kane. 

In the same field of vocal audio coding, Kane teaches a system to code the voice signal 
separately from the noise (non- voice) signal (Fig. 7, circuits 31-33 code the noise). Thus a voice 
may be separated from background music and both coded separately (col. 5, line 64-col. 6, line 
3). 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to modify the system of Saunders with the voice/non- voice separator of Kane, in order 
to increase the functionality of the system to be able to use audio data that does not already have 
vocal data separated from other audio data. 



Application/Control Number: 10/529,280 
Art Unit: 2626 



Page 12 



21 . Claim 7 is rejected under 35 U.S.C. 103(a) as being unpatentable over Saunders et al. 
(US 6,351,733), hereinafter referred to as Saunders, as applied to claim 1 above, and further in 
view of Newlin (US 5,774,857). 

22. With respect to claim 7, Saunders teaches everything claimed, as applied above (see 
claim 1); but Saunders does not teach a system as per claim 1, wherein said 
speech encoder is used in conjunction with a speech- to- text 
converter, and 

• said speech-to-text converter converting said speech 
component to a text component; and 

• said speech encoder encoding said text components and 
formatting said encoded text into a format suitable for 
transmission. 

However, the examiner contends that these concepts were well known in the art, as taught by 
Newlin. 

In the same field of endeavor of encoded audio and video transmission, Newlin teaches a 
system that accepts the input of an audio signal (Fig. 3, col. 11, lines 45-48) and processes the 
audio signal with a speech recognition subsystem to form a text representation of speech (Fig. 3, 
element 307, col. 11, lines 51-54). The text is then processed by the closed caption encoder (Fig. 
3, element 311) to convert it to a closed caption video format (col. 11, lines 54-61) to prepare it 
for mixing with the video signal for transmission (col. 11, lines 61-67). 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to modify the system of Saunders with the speech-to-text converter of Newlin, in order 
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to automatically generate a visual representation of speech without use of dedicated systems or 
user entry of the material to provide access for the hearing impaired (Newlin, col. 2, line 4-15). 

23. Claims 8, 15, and 18 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Saunders et al. (US 6,351,733), hereinafter referred to as Saunders, in view of Holmes et al. (US 
5,506,932), hereinafter referred to as Holmes. 

24. With respect to claim 8, Saunders teaches everything claimed, as applied above (see 
claim 1); in addition, Saunders teaches that audio tracks are time-synchronized and video-frame 
synchronized (col. 16, lines 15-24; claim 19) but Saunders does not specifically teach a 
system as per claim 1, wherein said embedded synchronization 
information is any of the following: timestamps, synchronization 
labels, media synchronization tags, synchronizing tokens, or 
wait -on- event commands . However, the examiner contends that this concept was well 
known in the art, as taught by Holmes. 

In a related field of endeavor of audio and video synchronization, Holmes teaches that 
audio data is synchronized to the video data on a frame-by-frame basis using a clock (col. 5, 
lines 20-23). Each frame is identified by a time stamp (col. 6, lines 23-24). 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to implement the synchronization taught by Saunders with the method using a time 
stamp as taught by Holmes, because the time stamp method is one of a finite number of methods 
known to be useful for audio-video data stream synchronization, as a person with ordinary skill 
has good reason to pursue the known options within his or her technical grasp. Using the known 



Application/Control Number: 10/529,280 Page 14 

Art Unit: 2626 

method would have had predictable results, so it would have been obvious to try the time stamp 
synchronization method to synchronize the data. 

25. With respect to claim 15, Saunders teaches everything claimed, as applied above (see 
claim 12); in addition, Saunders teaches that audio tracks are time-synchronized and video-frame 
synchronized (col. 16, lines 15-24; claim 19) but Saunders does not specifically teach a 
system as per claim 12, wherein said embedded synchronization 
information is any of the following: timestamps, synchronization 
labels, media synchronization tags, synchronizing tokens, or 
wait-on-event commands . However, the examiner contends that this concept was well 
known in the art, as taught by Holmes. 

In a related field of endeavor of audio and video synchronization, Holmes teaches that 
audio data is synchronized to the video data on a frame-by-frame basis using a clock (col. 5, 
lines 20-23). Each frame is identified by a time stamp (col. 6, lines 23-24). 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to implement the synchronization taught by Saunders with the method using a time 
stamp as taught by Holmes, because the time stamp method is one of a finite number of methods 
known to be useful for audio-video data stream synchronization, as a person with ordinary skill 
has good reason to pursue the known options within his or her technical grasp. Using the known 
method would have had predictable results, so it would have been obvious to try the time stamp 
synchronization method to synchronize the data. 
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26. With respect to claim 18, Saunders teaches everything claimed, as applied above (see 
claim 17); in addition, Saunders teaches that audio tracks are time-synchronized and video-frame 
synchronized (col. 16, lines 15-24; claim 19) but Saunders does not specifically teach a 
method as per claim 17, wherein said embedded synchronization 
information is any of the following: timestamps, synchronization 
labels, media synchronization tags, synchronizing tokens, or 
wait-on-event commands . However, the examiner contends that this concept was well 
known in the art, as taught by Holmes. 

In a related field of endeavor of audio and video synchronization, Holmes teaches that 
audio data is synchronized to the video data on a frame-by-frame basis using a clock (col. 5, 
lines 20-23). Each frame is identified by a time stamp (col. 6, lines 23-24). 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to implement the synchronization taught by Saunders with the method using a time 
stamp as taught by Holmes, because the time stamp method is one of a finite number of methods 
known to be useful for audio-video data stream synchronization, as a person with ordinary skill 
has good reason to pursue the known options within his or her technical grasp. Using the known 
method would have had predictable results, so it would have been obvious to try the time stamp 
synchronization method to synchronize the data. 

27. Claims 9 and 14 are rejected under 35 U.S.C. 103(a) as being unpatentable over Saunders 
et al. (US 6,351,733), hereinafter referred to as Saunders, in view of Rabowsky et al. (US 
5,680,512). 
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28. With respect to claim 9, Saunders teaches everything claimed, as applied above (see 
claim 1); but Saunders does not specifically teach a system as per claim 1 , 
wherein said music encoder is a MIDI-encoder or linear musical 
score not at ion . However, the examiner contends that this concept was well known in the 
art, as taught by Rabowsky. 

In the same field of endeavor of audio coding, Rabowsky teaches separate encoding of a 
voice and non- voice signal, which are combined to form the audio signal (coll, lines 49-60). 
The encoder for the non-voice signal (Fig. 1, element 12) may use MIDI (col. 5, lines 19-25, 
signal is applied to the encoder; col. 5, lines 43-57, MIDI elements are stored for reference in 
the encoder). 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to modify the audio coding system of Saunders with the MIDI encoder of Rabowsky, 
in order to increase the functionality of the coder to encode MIDI data. 

29. With respect to claim 14, Saunders teaches everything claimed, as applied above (see 
claim 12); but Saunders does not specifically teach a system as per claim 12, 
wherein said music encoder is a MDI-encoder or linear musical 
score not at ion . However, the examiner contends that this concept was well known in the 
art, as taught by Rabowsky. 

In the same field of endeavor of audio coding, Rabowsky teaches separate encoding of a 
voice and non- voice signal, which are combined to form the audio signal (coll, lines 49-60). 
The encoder for the non-voice signal (Fig. I, element 12) may use MIDI (col. 5, lines 19-25, 
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signal is applied to the encoder; col. 5, lines 43-57, MIDI elements are stored for reference in 
the encoder). 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to modify the audio coding system of Saunders with the MIDI encoder of Rabowsky, 
in order to increase the functionality of the coder to encode MIDI data. 

30. Claims 3, 4, and 20-23 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Saunders et al. (US 6,351,733), hereinafter referred to as Saunders, in view of Tsukagoshi (US 
6,104,861), and further in view of Ito et al. (US 2001/0012444), hereinafter referred to as Ito. 

31. With respect to claim 3, Saunders teaches everything claimed, as applied above (see 
claim 1); in addition, Saunders teaches that the audio may be synchronized with associated video 
(col. 10, lines 29-31; col. 16, lines 21-24) such as a motion picture, but Saunders does not 
explicitly teach a system as per claim 1, wherein said data further 
comprises a text component, a video component, and a graphics 
component, said system further comprising: 

• a text formatter transforming said text component into a 
format suitable for transmission and embedding 
synchronization information associated with said text 
component ; 

• a video encoder encoding said video component via a third 
encoding algorithm, said third encoding algorithm different 
from said first and second encoding algorithms; transforming 
said encoded video signal into a format suitable for 
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transmission; and embedding synchronization information 
associated with said video component; 

• a graphics encoder encoding said graphics component via a 
fourth encoding algorithm, said fourth encoding algorithm 
different from said first, second, and third encoding 
algorithms; transforming said encoded graphics into a format 
suitable for transmission; and embedding synchronization 
information associated with said graphics component; and 

• said multiplexer in (c) additionally multiplexing the output 
of said text formatter, said video encoder, and graphics 
encoder . 

However, the examiner contends that these concepts were well known in the art, as taught by 
Tsukagoshi and Ito. 

In the same field of endeavor of encoding and decoding of multiple types of data streams, 
Tsukagoshi teaches a system for encoding a video broadcast into multiple types of data streams 
for transmission. The system includes: a subtitle encoder (Fig. 9 A, element 57) that generates 
subtitles (col. 13, lines 16-29), selects data for compression and encodes it (col. 13, lines 30-55), 
and forwards the subtitle information to the multiplexer for multiplexing with the audio and 
video data to be transmitted (col. 14, lines 33-44); a video encoder (Fig. 9 A, element 52) which 
compresses digital video for video transmission (col. 13, lines 1-8) and synchronized with the 
subtitle data (col. 13, lines 5-8) and multiplexed with the other data by the multiplexer (col. 14, 
lines 33-44); as well as an audio encoder (Fig. 9A, lines 9-15). 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to use the audio encoding method of Saunders in the data encoding method of 
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Tsukagoshi because Tsukagoshi teaches an efficient way of transmitting video broadcast 
comprised of multiple data streams, as suggested by Saunders (col. 10, lines 29-31; col. 16, lines 
21-24). 

In the same field of endeavor of encoding and decoding of multiple types of data streams, 
Ito teaches a system for encoding a broadcast for transmission. The system comprises separate 
speech encoder (Fig. 18, element 5001) and a character object encoder (Fig. 18, element 5004), 
as well as a synthesized image object encoder (Fig. 18, element 5003, paragraphs [0087-0088]). 
The output of the encoder is multiplexed with the outputs of the other encoders and output as a 
bitstream for transmission (Fig. 18). A synthesized image may be a computer graphic 
(paragraph [0088]) such as a background object or weather information image (paragraph 
[0257]). 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to further modify the system of Saunders and Tsukagoshi with the additional graphic 
encoder of Ito, because the addition of graphics to the video would enable more information to 
be represented graphically to viewers, especially in such applications as a news program 
(paragraph [0257]). 

32. With respect to claim 4, Saunders in view of Tsukagoshi and Ito teach everything 
claimed, as applied above (see claim 3); in addition, Saunders does not but Tsukagoshi does 
teach a system as per claim 3, wherein said text component 
corresponds to subtitles associated with said video components 
(Fig. 9 A, col. 13, lines 16-29, information from character generator that is encoded is subtitles) . 
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It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to use the audio encoding method of Saunders in the data encoding method of 
Tsukagoshi because Tsukagoshi teaches an efficient way of transmitting video broadcast 
comprised of multiple data streams, as suggested by Saunders (Saunders, col. 10, lines 29-31; 
col. 16, lines 21-24). 

33. With respect to claim 20, Saunders teaches everything claimed, as applied above (see 
claim 19); in addition, Saunders teaches that the audio may be synchronized with associated 
video (col. 10, lines 29-31; col. 16, lines 21-24) such as a motion picture. However, while 
Saunders does not, Tsukagoshi teaches an article of manufacture as per claim 
19, wherein said plurality of components additionally comprises a 
video component, a text component, and a graphics component, said 
medium further comprising: 

a. in addition to de-multiplexing said data into speech and 
music component, computer readable program code de-multiplexing 
said video component, said text component, and said graphics 
component 

b. computer readable program code formatting said text 
component ; 

c . computer readable program code decoding said video 
component via a third decoding algorithm, said third decoding 
algorithm different from said first and second decoding 
algorithm; and 
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d. computer readable program code decoding said graphics 
component via a fourth decoding algorithm, said fourth decoding 
algorithm different from said first, second, and third decoding 
algorithm. 

However, the examiner contends that these concepts were well known in the art, as taught by 
Tsukagoshi and Ito. 

In the same field of endeavor of encoding and decoding of multiple types of data streams, 
Tsukagoshi teaches a system for decoding a video broadcast transmitted in multiple types of data 
streams. The system includes a demultiplexer to multiplex the signal into audio, video, and 
subtitle streams (Fig. 1, element 1, col. 4, lines 22-28). The subtitle data is decoded and mixed 
with the decoded video data (Fig. 1, col. 4, lines 52-56) and the video data is decoded and 
converted for display (Fig. 1, col. 4, lines 31-51). Each of the audio, video, and subtitle streams 
are decoded separately (Fig. 1). 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to use the audio decoding method of Saunders in the data decoding method of 
Tsukagoshi because Tsukagoshi teaches an efficient way of transmitting and decoding video 
broadcast comprised of multiple data streams, as suggested by Saunders (col. 10, lines 29-31; 

In the same field of endeavor of encoding and decoding of multiple types of data streams, 
Ito teaches decoding a transmitted broadcast. After the system has received and demultiplexed 
the data (Fig. 1, element 36), image data is decoded by the image decoding circuit (72) having a 
plurality of decoding units to decode respective image objects (paragraph [0266]). A type of 
image object is a synthesized image (paragraph [0084]), which may be a computer graphic 
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(paragraph [0088]) such as a background object or weather information image (paragraph 
[0257]). 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to further modify Saunders and Tsukagoshi with the additional graphic decoder of Ito, 
because the addition of graphics to the video would enable more information to be represented 
graphically to viewers, especially in such applications as a news program (paragraph [0257]). 

It is noted that Saunders in view of Tsukagoshi and Ito teaches the corresponding 
encoding method (see claim 3) and the decoding process is the reversal of the same sequence of 
steps. 

34. With respect to independent claim 21, Saunders teaches a method encoding 
data for transmission over a communication network, said data 
comprising speech, music ... components (col. 2, lines 15-18, audio program is 
produced so that the audio content is readily fabricated for transmission) , said method 
comprising the steps of: 

a. encoding said speech component via a first encoding 
algorithm (Fig. 9-12, col. 15, lines 21-24, signal is compressed with a speech-only codec) ; 

b. transforming said encoded speech signal into a format 
suitable for transmission (col. 16, lines 24-25, encoded data is ready for 
multiplexing and transmission) and embedding synchronization information 
associated with said speech component (col. 16, lines 15-24; claim 19; audio 
tracks are time-synchronized and video-frame synchronized to a video signal) ; 
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c . encoding said music component via a second encoding 
algorithm, said second encoding algorithm different from said 
first encoding algorithm (Fig. 9-12, col. 15, lines 21-30; non-speech signal is 
encoded using a general compression algorithm; col. 16, lines 9-12, two parallel streams are fed 
into two distinct compression algorithms) ; 

d. transforming said encoded music signal into a format 
suitable for transmission (col. 16, lines 24-25, encoded data is ready for 
multiplexing and transmission) ; and embedding synchronization information 
associated with said music component (col. 16, lines 15-24; claim 19; audio 
tracks are time-synchronized and video-frame synchronized to a video signal) ; 

wherein said first, second ... encoding algorithms are chosen 
to allow for low bit-rate compression of speech, music ... 
respectively (col. 15, lines 27-30, the distinction between compression using the speech- 
only codec and the general codec helps to reduce the required bandwidth) . 
Saunders also teaches that the compressed audio outputs are multiplexed so that the audio can be 
transmitted (Fig. 12, col. 16, lines 24-28; but Saunders does not teach 

e. encoding said video component via a third encoding 
algorithm, said third encoding algorithm different from said 
first and second encoding algorithms; 

f . transforming said encoded video signal into a format 
suitable for transmission and embedding synchronization 
information associated with said video component; 
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g. transforming a text component into a format suitable for 
transmission and embedding synchronization information associated 
with said text component; 

j. multiplexing said outputs of steps ... (f) [and] (g)... for 
transmission over said network, 

wherein said ... third ... encoding algorithms are chosen to 
allow for low bit-rate compression of ... video [and] text... 
respectively. 

However, the examiner contends that these concepts were well known in the art, as taught by 
Tsukagoshi. 

In the same field of endeavor of encoding and decoding of multiple types of data streams, 
Tsukagoshi teaches a system for encoding a video broadcast into multiple types of data streams 
for transmission. The system includes: a subtitle encoder (Fig. 9A, element 57) that generates 
subtitles (col. 13, lines 16-29), selects data for compression and encodes it (col. 13, lines 30-55), 
and forwards the subtitle information to the multiplexer for multiplexing with the audio and 
video data to be transmitted (col. 14, lines 33-44); a video encoder (Fig. 9 A, element 52) which 
compresses digital video for video transmission (col. 13, lines 1-8) and synchronized with the 
subtitle data (col. 13, lines 5-8) and multiplexed with the other data by the multiplexer (col. 14, 
lines 33-44); as well as an audio encoder (Fig. 9 A, lines 9-15). 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to use the audio encoding method of Saunders in the data encoding method of 
Tsukagoshi because Tsukagoshi teaches an efficient way of transmitting video broadcast 
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comprised of multiple data streams, as suggested by Saunders (col. 10, lines 29-31; col. 16, lines 
21-24). 

The combination of Saunders and Tsukagoshi does not teach: 

h. encoding said graphics component via a fourth encoding 
algorithm, said fourth encoding algorithm different from said 
first, second, and third encoding algorithm; 

i. transforming said encoded graphics signal into a format 
suitable for transmission; and embedding synchronization 
information associated with said graphics component; and 

j . multiplexing said outputs of steps ... (i) for transmission 
over said network, 

wherein said ... fourth encoding algorithms are chosen to 
allow for low bit-rate compression of ... graphics respectively. 

However, the examiner contends that these concepts were well known in the art, as taught by Ito. 

In the same field of endeavor of encoding and decoding of multiple types of data streams, 
Ito teaches a system for encoding a broadcast for transmission. The system comprises separate 
speech encoder (Fig. 18, element 5001) and a character object encoder (Fig. 18, element 5004), 
as well as a synthesized image object encoder (Fig. 18, element 5003, paragraphs [0087-0088]). 
The output of the encoder is multiplexed with the outputs of the other encoders and output as a 
bitstream for transmission (Fig. 18). A synthesized image may be a computer graphic 
(paragraph [0088]) such as a background object or weather information image (paragraph 
[0257]). 
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It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to further modify the system of Saunders and Tsukagoshi with the additional graphic 
encoder of Ito, because the addition of graphics to the video would enable more information to 
be represented graphically to viewers, especially in such applications as a news program 
(paragraph [0257]). 

35. With respect to claim 23, Saunders teaches everything claimed, as applied above (see 
claim 21); in addition, Saunders further teaches a method as per claim 21, 
wherein said network is any of the following: local area network, 
wide area network, the Internet, cellular network, storage area 
network, or wireless network (col. 9, lines 37-40; col. 25, lines 23-30; transmission 
may be an ISDN transmission to a PC modem) . 

36. Claim 22 is rejected under 35 U.S.C. 103(a) as being unpatentable over Saunders et al. 
(US 6,351,733), hereinafter referred to as Saunders, in view of Tsukagoshi (US 6,104,861) and 
Ito et al. (US 2001/0012444), hereinafter referred to as Ito, as applied to claim 21 above, and 
further in view of Holmes et al. (US 5,506,932), hereinafter referred to as Holmes. 

37. With respect to claim 22, Saunders in view of Tsukagoshi and Ito teaches everything 
claimed, as applied above (see claim 21); in addition, Saunders teaches that audio tracks are 
time-synchronized and video-frame synchronized (col. 16, lines 15-24; claim 19) but Saunders 
does not specifically teach a method as per claim 21, wherein said 
embedded synchronization information is any of the following: 
timestamps, synchronization labels, media synchronization tags, 
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synchronizing tokens, or wait-on-event commands. However, the examiner 
contends that this concept was well known in the art, as taught by Holmes. 

In a related field of endeavor of audio and video synchronization, Holmes teaches that 
audio data is synchronized to the video data on a frame-by-frame basis using a clock (col. 5, 
lines 20-23). Each frame is identified by a time stamp (col. 6, lines 23-24). 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to implement the synchronization taught by Saunders with the method using a time 
stamp as taught by Holmes, because the time stamp method is one of a finite number of methods 
known to be useful for audio-video data stream synchronization, as a person with ordinary skill 
has good reason to pursue the known options within his or her technical grasp. Using the known 
method would have had predictable results, so it would have been obvious to try the time stamp 
synchronization method to synchronize the data. 

Conclusion 

38. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

Imai et al. (JP 05-056007), Vaudrey et al. (US 6,442,278; US 6,985,594; US 6,311,155), 
Saunders et al. (US 2002/0040295), and Marko et al. (US 7,075,946) teach methods of encoding, 
transmitting, and decoding audio signals with separate speech and background audio signals. 

Sie et al. (US 5,534,941), Sherman et al. (US 6,535,269), Tsukagoshi (US 5,748,256) 
teach systems with multiple data streams. 
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Mead (US 6,088,484) and Fallon (US 6,624,761) teach using different methods of 
encoding for different types of data. 

39. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to GIN A W. LEE whose telephone number is (571)270-3 139. The 
examiner can normally be reached on Monday to Friday, 8:00 AM - 5:00 PM EST. 

40. If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Patrick Edouard can be reached on (571) 272-7603. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

41 . Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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